Current Empirical Research on Evaluation Utilization

This paper reviews empirical research conducted during the past 15 years on the use of evaluation results. Sixty-five studies in education, mental health, and social services are described in terms of their methodological characteristics, their orientation toward dependent and independent variables, and the relationships between such variables. A conceptual framework is developed that lists 12 factors that influence use; six of these factors are associated with characteristics of evaluation implementation and six with characteristics of decision or policy setting. The factors are discussed in terms of their influence on evaluation utilization, and their relative influence on various types of use is compared. The paper concludes with a statement about implications for research and practice. Ordinary knowledge provides the basis for decision and action in most organizations. Such knowledge, derived from practical experience, is usually widely shared, sensitive to context, and comprehensive. By contrast, knowledge derived from social science methods tends to be context independent and, of necessity, to be selective rather than comprehensive. At best, such knowledge supplements ordinary knowledge. The above argument (developed by Lindblom & Cohen, 1979) is a compelling one. On the one hand, it justifies the generation of knowledge using social science methods, and on the other, it imposes restrictions on the use of such knowledge. However, evaluators and others with a stake in social science have generally failed to recognize this argument, generating unrealistic expectations about the value of knowledge derived from social science methods and minimizing the application of such knowledge. Our purpose in this review is to assess what factors influence the use of evaluation data. Four questions guided our inquiry: What are the methodological characteristics of empirical studies for investigating evaluation use and its determinants? How have dependent variables been operationalized in these studies? What orientations toward independent variables have been adopted? What factors have been shown to affect the nature of use of evaluation results? This research is sponsored in part by the Ontarro Ministry of Colleges and Universities snd the Ontario Ministry of Education. The opinions expressed here, however, are solely the responsibility of the authors. The editorial assistance of Hugh Oliver and Mary Stager is gratefully acknowledged. Previous versions of this paper were presented at the 1985 annual meeting of the American Educational Research Association and the 1985 joint meeting of the Evaluation Network, Evaluation Research Society, and Canadian Evaluation Society.